Papers with robustness methods
FairLex: A Multilingual Benchmark for Evaluating Fairness in Legal Text Processing (2022.acl-long)
Copied to clipboard
| Challenge: | Using pre-trained language models, we evaluate performance group disparities while none of these techniques guarantee fairness, nor consistently mitigate group disparity. |
| Approach: | They present a benchmark suite of four datasets for evaluating the fairness of pre-trained language models and the techniques used to fine-tune them for downstream tasks. |
| Outcome: | The proposed methods show that performance group disparities are vibrant in many cases, while none of these techniques guarantee fairness, nor consistently mitigate group disparity. |
Can We Improve Model Robustness through Secondary Attribute Counterfactuals? (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent research has explored how models rely on spurious correlations and how counterfactual data augmentation (CDA) can mitigate such issues. |
| Approach: | They propose a context-aware methodology which takes into account the impact of secondary attributes on the model’s predictions and increases sensitivity for secondary attributes over reweighted counterfactually augmented data. |
| Outcome: | The proposed approach improves sliced accuracy on the original dataset by 7% compared to existing methods and provides guidelines to extend this to other tasks. |
READIN: A Chinese Multi-Task Benchmark with Realistic and Diverse Input Noises (2023.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for Chinese inputs often lack a realistic representation of real-world noises. |
| Approach: | They construct a Chinese multi-task benchmark with REalistic and Diverse input noises . they use pinyin input and speech input to recruit speakers from diverse dialects based on their inputs - a feature that is important for Chinese NLP benchmarks if it is implemented in real-world applications. |
| Outcome: | The proposed benchmarks are based on four different tasks and are designed to maximize diversity. |
Robertha: Eigenspectrum Regularized Attention for Robust Natural Language Understanding (2026.acl-long)
Copied to clipboard
| Challenge: | Existing robustness methods sacrifice clean performance or fail to generalize to higher corruption levels. |
| Approach: | They propose a mechanism that uses semantic patterns to pull corrupted embeddings toward correct representations by Eigenspectrum Regularization. |
| Outcome: | The proposed mechanism outperforms robustness methods on 13 GLUE and SuperGLUE tasks while maintaining competitive clean performance. |
When Punctuation Matters: A Large-Scale Comparison of Prompt Robustness Methods for LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are sensitive to subtle, non-semantic variations in prompt phrasing and formatting. |
| Approach: | They propose to evaluate 4 methods for improving prompt robustness within a unified experimental framework. |
| Outcome: | The proposed methods are compared to 8 models from Llama, Qwen and Gemma families and are generalized against multiple types of distribution shifts. |